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Instructions for Completing Consolidated State Application 

Accountability Workbook 

By January 31 , 2003, States must complete and submit to the Department this 
Consolidated State Application Accountability Workbook. We understand that some of 
the critical elements for the key principles may still be under consideration and may not 
yet be final State policy by the January 31 due date. States that do not have final 
approval for some of these elements or that have not finalized a decision on these 
elements by January 31 should, when completing the Workbook, indicate the status of 
each element which is not yet official State policy and provide the anticipated date by 
which the proposed policy will become effective. In each of these cases, States must 
include a timeline of steps to complete to ensure that such elements are in place by 
May 1 , 2003, and implemented during the 2002-2003 school year. By no later than May 
1 , 2003, States must submit to the Department final information for all sections of the 
Consolidated State Application Accountability Workbook. 

Transmittal Instructions 

To expedite the receipt of this Consolidated State Application Accountability Workbook, 
please send your submission via the Internet as a .doc file, pdf file, rtf or .txt file or 
provide the URL for the site where your submission is posted on the Internet. Send 
electronic submissions to conapp@ed.gov. 

A State that submits only a paper submission should mail the submission by express 
courier to: 

Celia Sims 

U.S. Department of Education 
400 Maryland Ave., SW 
Room 3W300 

Washington, D.C. 20202-6400 
(202) 401-0113 
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PART I: Summary of Required Elements for State Accountability 
Systems 

Instructions 

The following chart is an overview of States' implementation of the critical elements 
required for approval of their State accountability systems. States must provide detailed 
implementation information for each of these elements in Part II of this Consolidated 
State Application Accountability Workbook. 

For each of the elements listed in the following chart, States should indicate the current 
implementation status in their State using the following legend: 

F: State has a final policy, approved by all the required entities in the State (e.g., 

State Board of Education, State Legislature), for implementing this element in its 
accountability system. 

P: State has a proposed policy for implementing this element in its accountability 

system, but must still receive approval by required entities in the State (e.g., 
State Board of Education, State Legislature). 

W: State is still working on formulating a policy to implement this element in its 

accountability system. 
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Summary of Implementation Status for Required Elements of 
State Accountability Systems 



Status 



Pr nciple 1 : All Schools 



State Accountability System Element 



1 .1 Accountability system includes all schools and districts in the state. 

1 .2 Accountability system holds all schools to the same criteria. 

1 .3 Accountability system incorporates the academic achievement standards. 

1 .4 Accountability system provides information in a timely manner. 

1 .5 Accountability system includes report cards. 

1 .6 Accountability system includes rewards and sanctions. 



Principle 2: All Students 



Pr nciple 3: Method of AYP Determinations 



2.1 The accountability system includes all students 

2.2 The accountability system has a consistent definition of full academic year. 

2.3 The accountability system properly includes mobile students. 



3.1 Accountability system expects all student subgroups, public schools, and LEAs to reach 
proficiency by 2013-14. 

3.2 Accountability system has a method for determining whether student subgroups, public 
schools, and LEAs made adequate yearly progress. 

3.2a Accountability system establishes a starting point. 

3.2b Accountability system establishes statewide annual measurable objectives. 

3.2c Accountability system establishes intermediate goals. 



Pr nciple 4: Annual Decisions 



4.1 The accountability system determines annually the progress of schools and districts. 



STATUS Legend: 

F - Final state policy 

P - Proposed policy, awaiting State approval 
W - Working to formulate policy 
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Principle 5: Subarouo Accountabilit’ 




The accountability system includes all the required student subgroups. 

The accountability system holds schools and LEAs accountable for the progress of student 
subgroups. 

The accountability system includes students with disabilities. 

The accountability system includes limited English proficient students. 

The State has determined the minimum number of students sufficient to yield statistically 
reliable information for each purpose for which disaggregated data are used. 

The State has strategies to protect the privacy of individual students in reporting 
achievement results and in determining whether schools and LEAs are making adequate 
yearly progress on the basis of disaggregated subgroups. 



Principle 6: Based on Academic Assessments 



F 6.1 Accountability system is based primarily on academic assessments. 

Pr nciple 7: Additional Indicators 

F 7.1 Accountability system includes graduation rate for high schools. 

7.2 Accountability system includes an additional academic indicator for elementary and middle 

F schools. 

7.3 Additional indicators are valid and reliable. 

F_ 

Pr nciple 8: Separate Decisions for Readinq/Lanquaqe Arts and Mathematics 



F 8.1 Accountability system holds students, schools and districts separately accountable for 
reading/language arts and mathematics. 

Pr nciple 9: System Validity and Reliability 

F 9.1 Accountability system produces reliable decisions. 

9.2 Accountability system produces valid decisions. 

F_ 

9.3 State has a plan for addressing changes in assessment and student population. 

F_ 

Pr nciple 10: Participation Rate 

F 10.1 Accountability system has a means for calculating the rate of participation in the statewide 
assessment. 

1 0.2 Accountability system has a means for applying the 95% assessment criteria to student 
F subgroups and small schools. 

STATUS Legend: 

F - Final policy 

P - Proposed Policy, awaiting State approval 
W- Working to formulate policy 
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PART II: State Response and Activities for Meeting State 
Accountability System Requirements 



Instructions 

In Part II of this Workbook, States are to provide detailed information for each of the 
critical elements required for State accountability systems. States should answer the 
questions asked about each of the critical elements in the State's accountability system. 
States that do not have final approval for any of these elements or that have not 
finalized a decision on these elements by January 31 , 2003, should, when completing 
this section of the Workbook, indicate the status of each element that is not yet official 
State policy and provide the anticipated date by which the proposed policy will become 
effective. In each of these cases, States must include a timeline of steps to complete to 
ensure that such elements are in place by May 1, 2003, and implemented during the 
2002-2003 school year. By no later than May 1 , 2003, States must submit to the 
Department final information for all sections of the Consolidated State Application 
Accountability Workbook. 
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PRINCIPLE 1. A single statewide Accountability System applied to all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1.1 How does the State 
Accountability System 
include every public school 
and LEA in the State? 


Every public school and LEA is 
required to make adequate 
yearly progress and is included in 
the State Accountability System. 

State has a definition of “public 
school” and “LEA" for AYP 
accountability purposes. 

• The State Accountability 
System produces AYP 
decisions for all public 
schools, including public 
schools with variant grade 
configurations (e.g., K-12), 
public schools that serve 
special populations (e.g., 
alternative public schools, 
juvenile institutions, state 
public schools for the blind) 
and public charter schools. 

It also holds accountable 
public schools with no 
grades assessed (e.g., K- 
2). 


A public school or LEA is not 
required to make adequate 
yearly progress and is not 
included in the State 
Accountability System. 

State policy systematically 
excludes certain public schools 
and/or LEAs. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California currently has a comprehensive school accountability system in place that encompasses all 
schools, including public charter schools. California will utilize the federal measure of “proficient or above” 
in English-language arts (ELA) and mathematics to supplement its existing system, the cornerstone of 
which is the Academic Performance Index (API). 

The California Department of Education (CDE) will determine adequate yearly progress (AYP) based on 
the proportion of students scoring at “proficient or above” on the statewide assessments for all California 
local educational agencies (LEAs), schools, and numerically significant student subgroups within those 
LEAs and schools (see Critical Element 5.5). Consistent with federal law and regulation, if an LEA, 
school, or numerically significant student subgroup does not meet an annual measurable objective 
(AMO), i.e., the percentage of students scoring at “proficient or above,” based on the current year’s test 
results, California will average two or three years of test results to determine whether or not the LEA, 
school, or numerically significant student subgroup met the AMO. 

California will use its existing API as an additional academic indicator for all LEAs and schools, as 
provided for by the No Child Left Behind (NCLB) Act. Progress on the API for purposes of AYP will be 
defined as demonstrating a gain of one or more API points or meeting the annual API status target (see 
Attachment A). Originally, the API was only a school-level indicator. Since NCLB requires states to 
determine AYP for LEAs, the state now calculates APIs for all LEAs. A state API is also calculated and 
reported. 

Schools/LEAs with fewer than 1 00 valid scores 

A valid score in this context is a score that under federal law and regulation may be used to establish 
whether or not a student is proficient in ELA or mathematics. For a school, this would include test results 
from any student who has been continuously enrolled in the school from the preceding California Basic 
Educational Data System (CBEDS) date. For an LEA, this would include test results from any student 
who has been continuously enrolled in the LEA from the preceding CBEDS date. 

The CDE establishes AYP for schools/LEAs with fewer than 1 00 valid scores by applying a statistical test 
to one year’s worth of test results to achieve a 99 percent confidence level in AYP determination. 

Schools with no students enrolled in grades two through eight or ten 

The percent proficient or above in ELA and mathematics will be determined through the following 
procedure: 

Step One: Determine whether or not it is possible to pair the school with other schools in the 
same LEA and share test results from these schools (percent proficient or above) in ELA and 
mathematics. 

Step Two: If not, use test results from the California Standards Tests (CSTs in ELA and 
mathematics in grades nine and/or eleven so long as 100 percent of all students were required to 
take those tests. 

Step Three: If not, use LEA-wide results (percent proficient or above) in ELA and/or mathematics 
to determine the percent proficient or above for the school. If the school serves exclusively a 
special population (e.g., special education students), use the LEA-wide results for that student 
subgroup. For direct-funded charter schools, use the test results of the charter authorizer if 
results from the appropriate grades are available. Otherwise, use the county-level results. 
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Technical revisions to state law were made effective January 1 , 2005. (See Attachment B for a timeline of 
regulatory changes and legislative changes as well as other implementation activities.) 

Supporting Evidence: 

• The Academic Performance Index (API) as the “Other” Indicator (Attachment A) 

• API Description 

• Staff and Parents Assistance Packet for the 2002-03 API Growth 

• California’s timeline for NCLB implementation (see Attachment B) 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1.2 How are all public schools 
and LEAs held to the same 
criteria when making an AYP 
determination? 


All public schools and LEAs are 
systematically judged on the 
basis of the same criteria when 
making an AYP determination. 

If applicable, the AYP definition is 
integrated into the State 
Accountability System. 


Some public schools and LEAs 
are systematically judged on the 
basis of alternate criteria when 
making an AYP determination. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


To determine AYP, all public schools and LEAs will be evaluated on the basis of the percentage of 
students at or above proficient in ELA and mathematics. Schools and LEAs are also evaluated on the 
progress they make on the API. In addition, high schools are evaluated on the progress they make on the 
graduation rate. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .3 Does the State have, at a 
minimum, a definition of 
basic, proficient and 
advanced student 
achievement levels in 
reading/language arts and 
mathematics? 


State has defined three levels of 
student achievement: basic, 
proficient and advanced - 1 

Student achievement levels of 
proficient and advanced 
determine how well students are 
mastering the materials in the 
State’s academic content 
standards; and the basic level of 
achievement provides complete 
information about the progress of 
lower-achieving students toward 
mastering the proficient and 
advanced levels. 


Standards do not meet the 
legislated requirements. 



1 System of state achievement standards will be reviewed by the Standards and Assessments Peer 
Review. The Accountability Peer Review will determine that achievement levels are used in determining 
AYP. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California content standards 

California’s implementation and use of challenging academic content standards began in December 
1997, when the California State Board of Education (SBE) adopted content standards for ELA and 
mathematics. These standards contain coherent and rigorous content and specify what students are 
expected to know and be able to do. California’s world-class standards were developed for all students 
and can be attained by all students, given the appropriate standards-aligned instruction, sufficient time, 
and intervention when necessary. 

California’s assessment system 

The Standardized Testing and Reporting (STAR) program, first implemented in 1998, currently includes a 
norm-referenced test (California Achievement Test, Sixth Edition or CAT/6); the CSTs in ELA, 
mathematics, and history/social science (high school level); the Spanish Assessment of Basic Education 
(SABE/2); and the California Alternate Performance Assessment (CAPA) for students with severe 
cognitive disabilities. In addition to the assessments included in the STAR program, California administers 
the California High School Exit Exam (CAHSEE). The CAHSEE represents the core content that a high 
school graduate is expected to know and be able to demonstrate in ELA and mathematics. Results for 
the exam are reported separately by content area. The CAHSEE is required for all grade ten students. 

The SBE-approved performance levels on the CSTs at their meeting in February 2001 . Five performance 
levels were adopted: advanced, proficient, basic, below basic, and far below basic. Sensitivity to gains at 
the lower levels was one major concern that prompted the adoption of five performance levels, rather 
than the minimum of three required by NCLB. 

Elementary and middle schools 

Results from the CSTs will be used to determine the percentage of students scoring at the “proficient” 
level or above. 

High schools 

California will use results from the CAHSEE to establish AYP for high schools. California has identified 
three achievement levels required under NCLB for the CAHSEE as part of a technical process. CAHSEE 
test results are evaluated on the basis of pass/no pass for the high school graduation exit criterion and 
will continue to be evaluated in this manner for this criterion. 

Supporting Evidence: 

• California content standards 

• Information about the 2004 STAR program 

• Information about the California High School Exit Exam (CAHSEE) 

• California State Board of Education minutes from its February 7, 2001 , meeting where 
performance levels on the CSTs were approved (see Item 19) 

• California State Board of Education minutes from its January 8, 2003, meeting that describes 
which performance level will be used to judge “proficient” for NCLB 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .4 How does the State provide 
accountability and adequate 
yearly progress decisions 
and information in a timely 
manner? 


State provides decisions about 
adequate yearly progress in time 
for LEAs to implement the 
required provisions before the 
beginning of the next academic 
year. 

State allows enough time to 
notify parents about public school 
choice or supplemental 
educational service options, time 
for parents to make an informed 
decision, and time to implement 
public school choice and 
supplemental educational 
services. 


Timeline does not provide 
sufficient time for LEAs to fulfill 
their responsibilities before the 
beginning of the next academic 
year. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



AYP reporting 

On August 31 , 2005, prior to the beginning of the traditional school year, California will release all 
elements of the AYP determination (participation rates, AMOs, LEA and school-wide API, and graduation 
rate). These results will be based on the most current assessment data that the CDE has available. 

Identification as a Program Improvement school/LEA 

The CDE will require LEAs, upon receipt of AYP results, to notify the parents of all students assigned to a 
Title I school/LEA that fall into the category of Program Improvement (PI) of their school choice option. 
Parent notification under this plan will take place in time for alternative school assignments to be 
arranged, if requested. The CDE will confirm PI identification through the release of a PI Status Report 
after a school/LEA has had an opportunity to appeal its AYP results. 

Final school and LEA accountability reports and AYP determinations will be issued in January, once 
districts have submitted all demographic data corrections and the contractor has provided a revised data 
file. Once final accountability results are available, the CDE will revise the list of schools/LEAs identified 
for improvement to reflect any additions or deletions resulting from these final results. LEAs will then 
notify parents of the final results and make mid-year choice available in cases where the August AYP 
report did not identify schools for improvement. On the other hand, in cases where the CDE preliminarily 
identified a school/LEA for program improvement but deleted it from the final list, the CDE will inform the 
district, and the school will be relieved of prospective requirements. However, any school choice 
commitments (i.e., transportation costs) that were made based on preliminary identification will be 
honored for the balance of the school year. 

Supporting Evidence: 

• Title I Program Improvement Information for Schools and Districts 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .5 Does the State 


The State Report Card includes 


The State Report Card does not 


Accountability System 


all the required data elements 


include all the required data 


produce an annual State 


[see Appendix A for the list of 


elements. 


Report Card? 


required data elements]. 


The State Report Card is not 




The State Report Card is 
available to the public at the 
beginning of the academic year. 

The State Report Card is 
accessible in languages of major 
populations in the State, to the 
extent possible. 

Assessment results and other 
academic indicators (including 
graduation rates) are reported by 
student subgroups 


available to the public. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Accountability report cards have been a feature of California’s accountability system since 1988, when 
Proposition 98 required LEAs to produce them for each of their schools. School-level report cards are 
available on the Internet as links from the CDE Web site. As of January 2003, 99 percent of students in 
California’s public schools are in LEAs that have a link to report cards on the CDE Web site. 

Presently, the CDE produces a template of the report card, including data that are available from the 
state. LEAs then complete the report card by providing narrative sections and by supplying information 
available locally. As a result of NCLB, this template was expanded to include federally required LEA-level 
data. In most cases, these data will be an aggregation or “roll-up" of school-level data. LEA-level 
information is currently included in the School Accountability Report Card (SARC) templates. Additional 
LEA-required data and the state-level report card, representing an aggregation of the LEA-level data, 
were produced beginning in fall 2003. 

At the school, LEA, and state levels, the report card contains the required disaggregated results of 
student performance. Subgroups currently reported in the school report card include: (1) males and 
females; (2) economically disadvantaged; (3) limited English proficient; (4) students receiving migrant 
education services; (5) major racial/ethnic groups (African American/Black, Hispanic, Asian American, 
Pacific Islander, Filipino, American Indian/Alaska Native, White); and (6) all students. Performance results 
for students with disabilities were added with the 2002-03 reporting cycle. 

Some of the elements included in California’s SARC required modification to fully comply with the NCLB 
requirements. All changes were complete by October 2003 and were implemented with the 2002-03 
reporting cycle. 

Supporting Evidence: 

• School Accountability Report Card template that is made available to all LEAs 

• School Accountability Report Card home page for program information 

• California State Board of Education minutes from the March 12, 2003, meeting about the 
adoption of the report card template 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 .6 How does the State 


State uses one or more types of 


State does not implement 


Accountability System 


rewards and sanctions, where 


rewards or sanctions for public 


include rewards and 


the criteria are: 


schools and LEAs based on 


sanctions for public schools 
and LEAs? 2 


• Set by the State; 

• Based on adequate yearly 
progress decisions; and, 

• Applied uniformly across 
public schools and LEAs. 


adequate yearly progress. 



2 The state must provide rewards and sanctions for all public schools and LEAs for making adequate 
yearly progress, except that the State is not required to hold schools and LEAs not receiving Title I funds 
to the requirements of Section 1116 of NCLB [Section 200.1 2(b)(40)]. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Current state law provides award programs, both monetary and nonmonetary, for schools that meet their 
API targets and intervention programs for those schools that do not. Each of the state award and 
intervention programs is available to all California schools, regardless of Title I funding, so long as the 
schools meet eligibility requirements in state law and regulation. 

The monetary award programs include the Governor’s Performance Award Program and the Certificated 
Staff Performance Incentive Program. These programs are currently unfunded and unlikely to be funded 
in the foreseeable future. Nonmonetary programs include the state California School Recognition 
Program and the federal NCLB-Blue Ribbon Schools Program and the Title I Academic Achieving 
Schools Programs. 

California has two statewide intervention programs in place: the Immediate Intervention/Underperforming 
Schools Program and the High Priority Schools Grant Program. Schools that are eligible for either of 
these programs are the lowest performing in the state; participation is voluntary. In either program, 
schools are subject to state sanctions in the event that they fail to demonstrate significant growth on the 
API during the school action plan implementation period. Together, these two state programs include 
about 1 ,700 schools in various stages. Also, schools that receive Title I funds are identified for PI if they 
fail to make AYP for two consecutive years. PI schools that continue to fail to make AYP are subject to 
corrective action, as required by federal law. 

Discussions are continuing to align current state intervention programs with the requirements of NCLB. 
Since California has well-established state intervention programs with a significant number of schools as 
participants, realignment will require extensive legislative and regulatory changes. These changes, in 
turn, will require input from a number of constituencies. 

Also, the transition from the current amalgam of state and federal programs to an aligned system will take 
some time in order to guarantee schools and districts have adequate notice and fair opportunities to make 
improvements before sanctions are invoked. 

A proposed strategy for aligning state and federal policy for underperforming schools is being finalized. 
This proposal is intended to serve as the basis of legislation to align state programs with federal 
requirements. It is anticipated that this alignment process will take at least two years due to its complexity. 

Supporting Evidence: 

• California Education Code Section 52057 (Governor's Performance Awards program) and 
California Education Code Section 44650 (Certificated Staff Performance Incentive Act) for 
program descriptions of the two state-level monetary awards programs 

• California School Recognition Program description 

• Achieving Schools program eligibility criteria for 2003 (see Attachment C) 

• Information on the California National Blue Ribbon Schools Program 

• News Release About 2002-03 California's National Blue Ribbon Schools Nominees 

• Awards Program Alignment Plan (Attachment D) 

• California Education Code Section 52053-52053.53 (Immediate Intervention/Underperforminq 
Schools Program) and California Education Code Section 52055.600-52055.660 (High Priority 
Schools Grant Program) for program descriptions of the state-level intervention programs 

• Comparison of state and federal intervention programs (Attachment E) 
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PRINCIPLE 2. All students are included in the State Accountability System. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.1 How does the State 
Accountability System 
include all students in the 
State? 


All students in the State are 
included in the State 
Accountability System. 

The definitions of “public school” 
and “LEA" account for all 
students enrolled in the public 
school district, regardless of 
program or type of public school. 


Public school students exist in 
the State for whom the State 
Accountability System makes no 
provision. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California Education Code (EC) Section 60640(b) requires each school district, charter school, and 
county office of education to administer the STAR Program assessments to each of its pupils in grades 
two through eleven, unless the pupil is excused by a parent request (allowable by California law - see 
California EC Section 6061 5). The CDE has developed and implemented an alternate assessment for 
children with severe disabilities who cannot take part in the regular STAR assessments - the CAPA. All 
students with disabilities, unless excused by parent request, are required to participate in either the STAR 
assessments or the CAPA. At the high school level, California EC Section 60851 (b) requires each grade 
ten student to take the CAHSEE. The STAR, CAPA, and CAHSEE results will be merged for AYP 
reporting purposes, as appropriate. 

Test results summarized according to the federal AYP criterion (percent proficient or above in ELA and 
mathematics) will be reported at the school level for all students who have been continuously enrolled in 
the school since the beginning of the academic year; at the district level for all students who have been 
continuously enrolled in the district since the beginning of the academic year; and for all students at the 
state level. These definitions and procedures will apply to all schools and LEAs statewide. Test scores for 
students with disabilities in county office special education schools or programs will be assigned back to 
the district of residence for LEA accountability. This is consistent with the provisions of Section 9101(1) of 
Title IX of NCLB. 

Originally, California held a school accountable for the performance of any student who had been enrolled 
in the school district for a full academic year. Under NCLB, a school is accountable only if the student has 
been enrolled in the school for a full academic year. In order to align the API system with the provisions of 
NCLB, California has amended the relevant California EC Section so that a school is accountable only for 
the performance of a student who has been enrolled in the school for a full academic year. A district is 
held accountable for all students who were enrolled in the district for a full academic year. The 
performance results of students not enrolled in any one district for a full academic year are aggregated to 
the state level. 

Supporting Evidence: 

• California Education Code Section 60640(b) regarding student participation in the Standardized 
Testing and Reporting (STAR) assessment program 

• California Education Code Section 60851(b) regarding student participation in the California High 
School Exit Exam 

• California Education Code Section 60615 that describes the allowance for parental waivers to 
excuse students from the statewide testing program 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.2 How does the State define 


The State has a definition of “full 


LEAs have varying definitions of 


“full academic year” for 
identifying students in AYP 


academic year” for determining 
which students are to be included 


“full academic year.” 


decisions? 


in decisions about AYP. 

The definition of full academic 
year is consistent and applied 
statewide. 


The State’s definition excludes 
students who must transfer from 
one district to another as they 
advance to the next grade. 

The definition of full academic 
year is not applied consistently. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Currently, California uses the date on which LEAs are required to submit information to CBEDS as the 
beginning of the academic year for reporting purposes. This date occurs annually in October, generally on 
the first Wednesday. If a student is continuously enrolled in a school from that date to the date of testing 
in the spring, California considers the student to have been enrolled in that school for a “full academic 
year.” The same criterion will be used for LEAs. 

“Continuous enrollment in the school (or LEA) from the CBEDS date” is defined as: 

The student did not withdraw or was not dropped from the school’s (or LEA’s) enrollment any time 
between the CBEDS census date and the first day of testing. 

California law about school mobility for the API has been changed to reflect the criteria outlined above. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING STATUTORY 
REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


2.3 How does the State 
Accountability System 
determine which students 
have attended the same 
public school and/or LEA for 
a full academic year? 


State holds public schools 
accountable for students who 
were enrolled at the same public 
school for a full academic year. 

State holds LEAs accountable for 
students who transfer during the 
full academic year from one 
public school within the district to 
another public school within the 
district. 


State definition requires students 
to attend the same public school 
for more than a full academic 
year to be included in public 
school accountability. 

State definition requires students 
to attend school in the same 
district for more than a full 
academic year to be included in 
district accountability. 

State holds public schools 
accountable for students who 
have not attended the same 
public school for a full academic 
year. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


Information about the enrollment of students is collected on the Student Answer Document (SAD) that is 
completed at the time of testing for the STAR tests, the CAHSEE, and the CAPA. A SAD is completed not 
only for a student who actually participates in the assessment but also for a student who is enrolled in one 
of the grades tested and does not take the assessment. The SAD for all three tests includes two 
questions designed to address student mobility: 

• Has the student been continuously enrolled at the school since the CBEDS date? 

• Has the student been continuously enrolled in the district since the CBEDS date? 

This is the method by which California determines which students will count at the school, district, or state 
level. 

Supporting Evidence: 

• Student Answer Document (SAD) for the 2004 testina cycle 

• Information for Codina Test Documents (2004 testina cycle) 
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PRINCIPLE 3. State definition of AYP is based on expectations for growth in 
student achievement that is continuous and substantial, such that all students 
are proficient in reading/language arts and mathematics no later than 2013-2014. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.1 How does the State’s 

definition of adequate yearly 
progress require all students 
to be proficient in 
reading/language arts and 
mathematics by the 201 3- 
2014 academic year? 


The State has a timeline for 
ensuring that all students will 
meet or exceed the State’s 
proficient level of academic 
achievement in reading/language 
arts 3 and mathematics, not later 
than 2013-2014. 


State definition does not require 
all students to achieve 
proficiency by 2013-2014. 

State extends the timeline past 
the 201 3-2014 academic year. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


At their January 8, 2003, meeting, the SBE adopted a definition of AYP for both ELA and mathematics. 
The definition is based on a statewide performance goal of all students scoring at or above the “proficient” 
level on the standards-based statewide assessments by the required deadline of 2013-2014. 

Performance on the CSTs in ELA and mathematics is used to evaluate AYP for students in grades two 
through eight; the CAHSEE is used to evaluate performance of students in grades ten through twelve. 

Supporting Evidence: 

• California State Board of Education minutes from its January 8, 2003, meetinq 



3 If the state has separate assessments to cover its language arts standards (e.g., reading and writing), 
the State must create a method to include scores from all the relevant assessments. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2 How does the State 
Accountability System 
determine whether each 
student subgroup, public 
school and LEA makes 
AYP? 


For a public school and LEA to 
make adequate yearly progress, 
each student subgroup must 
meet or exceed the State annual 
measurable objectives, each 
student subgroup must have at 
least a 95% participation rate in 
the statewide assessments, and 
the school must meet the State’s 
requirement for other academic 
indicators. 

However, if in any particular year 
the student subgroup does not 
meet those annual measurable 
objectives, the public school or 
LEA may be considered to have 
made AYP, if the percentage of 
students in that group who did 
not meet or exceed the proficient 
level of academic achievement 
on the State assessments for that 
year decreased by 1 0% of that 
percentage from the preceding 
public school year; that group 
made progress on one or more of 
the State’s academic indicators; 
and that group had at least 95% 
participation rate on the 
statewide assessment. 


State uses different method for 
calculating how public schools 
and LEAs make AYP. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



In California, in order to make AYP, a public school or LEA must meet or exceed the state’s AMOs in ELA 
and mathematics, demonstrate progress on the API as the other indicator, and demonstrate a 
participation rate in each applicable assessment of 95 percent or higher. Additionally, each student 
subgroup within a public school or LEA will have to meet or exceed the state’s AMOs in ELA or 
mathematics and have a participation rate of 95 percent or more in each assessment, if the subgroup 
meets the definition of a numerically significant subgroup (see Critical Element 5.5). 

If the LEA, school, or student subgroup does not meet its AMOs based on the current year’s test results, 
the last two or three year’s test results will be aggregated in order to determine whether or not the LEA, 
school, or student subgroups meet the AMOs. 

In the event that an LEA, school, or student subgroup does not meet its AMOs, the LEA, school, or 
subgroup may be considered to have made AYP if all of the following conditions are met: 

• The percentage of students in a LEA, school, or subgroup who did not meet or exceed the 
“proficient” level on the applicable assessments decreased by 10 percent of that percentage 
from the preceding school year. A confidence interval of 75 percent will be applied in making this 
determination. 

• The LEA, school, or subgroup demonstrated growth on the API of at least one point from the 
preceding school year or attained the annual API status target. (California will use the API as an 
additional academic indicator as per Critical Element 1.1.) 

• For high schools, the LEA, school, or student subgroup demonstrated progress in increasing its 
graduation rate. 

• The LEA, school, or subgroup had at least a 95 percent participation rate on the applicable 
assessment, (see Critical Element 10.1 for California’s method for determining participation rate.) 

California will identify for PI any school that fails to make AYP for two consecutive years in the same 
content area or the same additional indicator (API for all schools, graduation rate for high schools). For 
example, a school that had a subgroup fail to meet or exceed the AMO in ELA in Year 1 and in Year 2 
would be identified for PI. A school that failed to meet or exceed the AMO in mathematics in Year 1 , but in 
Year 2 failed to meet or exceed the AMO in ELA would not be identified for PI. A school that failed to 
meet the API criteria for two consecutive years would be identified for PI. A high school that failed to meet 
the graduation rate criteria for two consecutive years would be identified for PI. 

California will initially identify for PI any district that fails to make AYP for two consecutive years in the 
same content area or on the same additional indicator (API for all districts, graduation rate for districts 
with high school students). For those districts that missed the AMOs in the same content area for two 
consecutive years, California will apply a second criterion: did any grade span within the district 
(elementary, middle, and high school) meet the grade span AMO in either of the two years in question? If 
yes, the district will not be identified for PI. 



25 





CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2a What is the State’s starting 
point for calculating 
Adequate Yearly 
Progress? 


Using data from the 2001-2002 
school year, the State 
established separate starting 
points in reading/language arts 
and mathematics for measuring 
the percentage of students 
meeting or exceeding the State’s 
proficient level of academic 
achievement. 

Each starting point is based, at a 
minimum, on the higher of the 
following percentages of students 
at the proficient level: (1 ) the 
percentage in the State of 
proficient students in the lowest- 
achieving student subgroup; or, 
(2) the percentage of proficient 
students in a public school at the 
20 th percentile of the State’s total 
enrollment among all schools 
ranked by the percentage of 
students at the proficient level. 

A State may use these 
procedures to establish separate 
starting points by grade span; 
however, the starting point must 
be the same for all like schools 
(e.g., one same starting point for 
all elementary schools, one same 
starting point for all middle 
schools...). 


The State Accountability System 
uses a different method for 
calculating the starting point (or 
baseline data). 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Using 2002 STAR data, California has established separate starting points in ELA and mathematics for 
measuring the percentage of students at or above proficient for grades two through eight: California’s 
starting point for ELA is 1 3.6 percent at or above “proficient”; for mathematics the starting point is 1 6.0 
percent at or above “proficient.” Using 2001 and 2002 CAHSEE data, for high schools the corresponding 
starting points are 1 1 .2 percent at or above “proficient” for ELA and 9.6 percent in mathematics. All of the 
starting points are based on the percentage of students at “proficient” or above in a public school at the 
twentieth percentile of California’s total enrollment among all schools ranked by the percentage of 
students at the proficient or advanced levels. In all cases, this produces the higher percentage from the 
two methods prescribed by NCLB. 

Supporting Evidence: 

• Starting Point Calculation for Grades 2-8 (Attachment G) 

• Starting Point Calculation for Grade 1 0 (Attachment H) 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2b What are the State’s annual 
measurable 

objectives for determining 
adequate yearly progress? 


State has annual measurable 
objectives that are consistent 
with a state’s intermediate goals 
and that identify for each year a 
minimum percentage of students 
who must meet or exceed the 
proficient level of academic 
achievement on the State’s 
academic assessments. 

The State’s annual measurable 
objectives ensure that all 
students meet or exceed the 
State’s proficient level of 
academic achievement within the 
timeline. 

The State’s annual measurable 
objectives are the same 
throughout the State for each 
public school, each LEA, and 
each subgroup of students. 


The State Accountability System 
uses another method for 
calculating annual measurable 
objectives. 

The State Accountability System 
does not include annual 
measurable objectives. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


California has established separate ELA and mathematics statewide AMOs that identify a minimum 
percentage of students that must meet the “proficient” level of academic achievement. AMOs will utilize 
the same percent proficient as the most recent intermediate goal, or, in the beginning, the starting point 
(see Critical Element 3.2c). The ELA and mathematics AMOs will be applied to each school and LEA, as 
well as to each subgroup at the school, LEA, and statewide levels to determine AYP status. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


3.2c What are the State’s 


State has established 


The State uses another method 


intermediate goals for 


intermediate goals that increase 


for calculating intermediate goals. 


determining adequate 


in equal increments over the 




yearly progress? 


period covered by the State 


The State does not include 




timeline. 

• The first incremental 

increase takes effect not 
later than the 2004-2005 
academic year. 

• Each following incremental 

increase occurs within 
three years. 


intermediate goals in its definition 
of adequate yearly progress. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California will establish separate ELA and mathematics intermediate goals statewide that increase in 
equal increments over the 1 2-year timeline. There will be seven intermediate goals total. For grades two 
through eight, these goals are: 



ELA 


Year 


Math 






13.6% 


2002 


16.0% 


(Starting Points) 


24.4% 


2005 


26.5% 


(Intermediate Goals) 


35.2% 


2008 


37.0% 






46.0% 


2009 


47.5% 






56.8% 


2010 


58.0% 






67.6% 


2011 


68.5% 






78.4% 


2012 


79.0% 






89.2% 


2013 


89.5% 


1 


r 


100.0% 2014 


1 00.0% 


(Final Goals) 



This schedule of intermediate goals will result in all students in grades two through eight meeting or 
exceeding the proficient level of academic achievement in ELA and mathematics not later than 2013- 
2014, as required by law. 

Since the peer review, a similar schedule of intermediate goals was developed for high schools 
(Attachments N and O) as well as districts (Attachments L, M, N, O, P, and Q). The schedule for districts 
that enroll students in grades nine through twelve only is identical to the high school schedule. The 
schedule for districts that enroll students in grades two through eight only is identical to the elementary 
school schedule. The schedules for districts that enroll both primary and secondary students are an 
average of the high school and elementary school schedules. In applying a grade span analysis when 
identifying LEAs for PI, the CDE will apply the goals for grades two through eight to the elementary and 
middle school grade spans (two through five and six through eight) and the grade ten goal for the high 
school grade span. 

These intermediate goals are consistent with the expectation that the strongest academic gains in 
schools and districts are likely to occur in later years, after teachers are given time to align their 
instruction with academic content standards, after districts are given the opportunity to increase their 
capacity to support needed reforms, and after there is a highly qualified teacher in every California 
classroom. This is particularly true for low-performing schools in California where students are being 
asked to reach performance levels that are especially rigorous (see Critical Element 1 .3). 

Supporting Evidence: 

• Twelve-year timeline for ELA with annual measurable objectives and intermediate goals for 
elementary schools, middle schools, elementary districts, and middle school districts (Attachment 

I) 

• Twelve-year timeline for mathematics with annual measurable objectives and intermediate goals 
for elementary schools, middle schools, elementary districts, and middle school districts 
(Attachment J) 

• Twelve-year timeline for ELA with annual measurable objectives and intermediate goals for high 
schools and high school districts (Attachment K) 

• Twelve-year timeline for mathematics with annual measurable objectives and intermediate goals 
for high schools and high school districts (Attachment L) 

• Twelve-year timeline for ELA with annual measurable objectives and intermediate goals for 
unified districts and high school districts with grades seven and/or eight (Attachment M) 

• Twelve-year timeline for mathematics with annual measurable objectives and intermediate goals 

for unified districts and high school districts with grades seven and/or eight (Attachment N) 
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PRINCIPLE 4. State makes annual decisions about the achievement of all public 
schools and LEAs. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


4.1 How does the State 
Accountability System 
make an annual 
determination of whether 
each public school and LEA 
in the State made AYP? 


AYP decisions for each public 
school and LEA are made 
annually. 4 


AYP decisions for public schools 
and LEAs are not made annually. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The CDE made annual AYP decisions for all schools receiving Title I, Part A funds preceding the 
enactment of NCLB. Expanding this established practice, the CDE now makes an annual AYP 
determination for all schools and LEAs. For schools/LEAs, this determination results from whether or not 
the schools/LEAs met the AMOs in ELA and mathematics, the participation rate criteria, and 
demonstrated progress on the additional indicator(s). The AMOs and participation rate criteria are applied 
overall and to each numerically significant student subgroup within the school or LEA. The CDE will make 
use of the methodologies identified in Critical Elements 1.1 and 1 0.1 for schools/LEAs with fewer than 
100 test scores and/or 100 students enrolled in the grades tested. The 99 percent confidence interval test 
is not applied to numerically significant subgroups with 50 to 99 test scores. 

Supporting Evidence: 

• Title I District and School Profiles that show the status of all 2003 Title I schools in the state 

• Title I District and School Profiles that show the status of all 2003 Title I schools prior to 2003 in 
the state 

• Title I Program Improvement Information for Schools and Districts 



4 Decisions may be based upon several years of data, and data may be averaged across grades within a 
public school [Section 1111 (b)(2)(J)]. 
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PRINCIPLE 5. All public schools and LEAs are held accountable for the 
achievement of individual subgroups. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.1 How does the definition of 
adequate yearly progress 
include all the required 
student subgroups? 


Identifies subgroups for defining 
adequate yearly progress: 
economically disadvantaged, 
major racial and ethnic groups, 
students with disabilities, and 
students with limited English 
proficiency. 

Provides definition and data 
source of subgroups for adequate 
yearly progress. 


State does not disaggregate data 
by each required student 
subgroup. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California currently disaggregates test results by each of the student subgroups required by NCLB per 
demographic information collected on the SAD at the time of testing. The CDE will use these results to 
determine the percent of students at the “proficient” level or above by subgroup for the annual AYP 
determination for all schools and LEAs in the state. 

Currently, state law requires that numerically significant racial/ethnic and SED subgroups will demonstrate 
“comparable improvement” in order for the school to meet its API growth target. (Note: For accountability 
purposes, California defines a SED subgroup rather than an economically disadvantaged subgroup. A 
student is included in the SED subgroup if they participate in the Free and Reduced-Price Meals Program 
or if the highest level of education of either of the student’s parents is less than a high school diploma. 

This definition was chosen for the API system in 1 999 because of the non-universality of the free and 
reduced-price meals indicator, particularly at the high school level, and because parental education is 
generally accepted as a better predictor of students performance on a standardized test than participation 
in Free and Reduced Price Meals Program, regardless of the student’s grade.) California will use this 
definition to determine whether or not the SED subgroup under NCLB met AMOs and therefore made 
AYP. 

In order to align the API with NCLB requirements, California has added two additional student subgroups 
required by NCLB, students with disabilities and students with limited English proficiency, to the 
subgroups required to demonstrate “comparable improvement.” This addition to state law was effective 
January 1 , 2005 

Supporting Evidence: 

• Sample report from the Standardized Testing and Reporting (STAR) system that shows 
disaggregated data for all NCLB required subgroups 

• Sample report from the California High School Exit Exam (CAHSEE) that shows disaggregated 
data for subgroups 

• Student Answer Document (SAD) for the 2004 testing cycle 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.2 How are public schools 
and LEAs held 
accountable for the 
progress of student 
subgroups in the 
determination of adequate 
yearly progress? 


Public schools and LEAs are held 
accountable for student subgroup 
achievement: economically 
disadvantaged, major ethnic and 
racial groups, students with 
disabilities, and limited English 
proficient students. 


State does not include student 
subgroups in its State 
Accountability System. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Numerically significant student subgroups will be expected to meet AMOs in ELA and mathematics and to 
participate in student assessments at a rate of 95 percent or higher in order for a public school or LEA to 
demonstrate AYP. 

In the event that an LEA, school, or student subgroup does not meet its AMOs, the LEA, school or 
subgroup may be considered to have made AYP if all of the following conditions are met: 

• The percentage of students at a LEA, school, or subgroup who did not meet or exceed the 
“proficient” level on the applicable assessments decreased by 1 0 percent of that percentage from 
the preceding school year. A confidence interval of 75 percent will be applied in making this 
determination. 

• The LEA, school, or subgroup demonstrated growth on the API of at least one point from the 
preceding school year or attained the annual API status target. (California will use the API as an 
additional academic indicator as per Critical Element 1.1.) 

• For high schools, the LEA, school, or student subgroup demonstrated progress in increasing its 
graduation rate as defined in Critical Element 7.1 . 

• The LEA, school, or subgroup had at least a 95 percent participation rate on the applicable 
assessment as defined in Critical Element 10.1. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.3 How are students with 
disabilities included in the 
State’s definition of 
adequate yearly progress? 


All students with disabilities 
participate in statewide 
assessments: general 
assessments with or without 
accommodations or an alternate 
assessment based on grade level 
standards for the grade in which 
students are enrolled. 

State demonstrates that students 
with disabilities are fully included 
in the State Accountability 
System. 


The State Accountability System 
or State policy excludes students 
with disabilities from participating 
in the statewide assessments. 

State cannot demonstrate that 
alternate assessments measure 
grade-level standards for the 
grade in which students are 
enrolled. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Students with disabilities within the grades tested (grades two through eight and ten) participate in 
California’s statewide assessment program by taking either the general assessment with or without 
accommodations/modifications or the alternate assessment. The majority of students with disabilities 
participate in the general assessment, but those with significant cognitive disabilities may be eligible to 
participate in the CAPA. Results from students with disabilities who participate in the general statewide 
assessment (i.e. STAR) will be summarized in the same fashion as any other students. 

Consistent with the transition policy included in the evaluation of our January 2003 Workbook, in 2003 
California treated the five CAPA performance levels as equal to the five performance levels used for the 
CSTs for summarizing LEA and school performance. In other words, a score (performance level) on the 
alternate assessment holds the same value as a score (performance level) for the STAR or CAHSEE. 
Beyond 2003, the scores for students with disabilities who take the alternate assessment will be 
included in the assessment data in the accountability system within the parameters defined by federal 
statute and regulations. 

Starting in 2004, the CDE gave any LEA that exceeds the one percent cap on the application of 
alternate standards and assessments the opportunity to request an exception. The exception shall be 
based on criteria similar to those that the U.S. Department of Education (ED) uses to evaluate state 
requests for exception. If the CDE does not grant the request, it will adjust the results of the LEA by 
treating the appropriate number of scores as “not proficient” for accountability purposes only. This will 
not affect the individual student report received by parents. 

In making this adjustment, the CDE will first consider the scores of students enrolled in LEA-wide 
programs who took the CAPA. If this does not bring the LEA down to the one percent cap, the CDE will 
then allocate the remaining non-proficient scores among schools in the LEA that have students who 
scored proficient or advanced on the CAPA. CDE will then recalculate the percentage of students at 
proficient or above for the LEA, school, and subgroup levels. The State, not the individual LEA, will 
determine which CAPA scores will be adjusted for accountability purposes. 

On grounds of validity and reliability, the CDE will not apply the one percent cap to small LEAs with 
fewer than 1 0 scores in ELA or mathematics or with fewer than five proficient or advanced CAPA scores 
in a content area. 

In addition to the students with the most significant cognitive disabilities, constituting about one percent 
of the total student population, research demonstrates that other students with disabilities, approximately 
two percent of the total student population, are not able to meet grade-level standards, even after 
application of the best-designed instructional interventions. For this second group of students, California 
will develop modified achievement standards as well as alternate assessments based on those 
standards. These assessments will be aligned with grade-level content standards. 

Since these assessments are currently under development, California will continue to exercise the 
interim flexibility offered by the USED in making AYP determinations for the students with disabilities 
subgroup as summarized in Secretary Spelling’s correspondence of May 10, 2005. Specifically, 

California will apply transition Option #1 in making AYP determinations based on assessment results 
from the 2005-2006 school year: 

1 . The CDE will calculate what 2.0 percent of the total number of students assessed within 
California equates to solely within the students with disabilities subgroup by dividing 2.0 by 
the percentage of students who have disabilities. This numerical constant will be the basis 
for exercising the interim flexibility. In 2005, this constant was 20%. 
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2. The CDE will identify all LEAs and schools that did not make AYP in the 2005-2006 school 
year based solely on the basis of the students with disabilities subgroup. 

3. The CDE will calculate the adjusted percent proficient for the students with disabilities 
subgroup for each of these LEAs and schools. This adjustment is equal to the sum of the 
actual percentage of proficient scores of this subgroup plus the proxy from # 1 above. 

• If the adjusted rate for the LEA or school’s student with disabilities subgroup meets 
or exceeds 2006 AMO, the district or school will be considered to have made AYP 
for the 2005-2006 school year. 

• If the adjusted rate for the LEA or school’s students with disabilities subgroup does 
not meet the AMO, the district or school did not make AYP for the 2005-2006 school 
year. 

4. This process will be applied for ELA and mathematics separately. The unadjusted percent 
proficient will be reported to parents and the public. 

Test scores for students with disabilities in county office special education schools or programs will be 
assigned back to the district of residence for LEA accountability. This is consistent with the provisions of 
Section 91 01 (1 ) of Title IX of NCLB. 

Supporting Evidence: 

• California Code of Regulations Section 853(c) that describes California’s policy for testing 
students with disabilities (see Title 5, Division 1 , Chapter 2, Subchapter 3.75, Article 2, Section 
853(c)) 

• Special Education Accommodations/Modifications for California Statewide Assessments 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.4 How are students with 


All LEP student participate in 


LEP students are not fully 


limited English proficiency 


statewide assessments: general 


included in the State 


included in the State’s 
definition of adequate 
yearly progress? 


assessments with or without 
accommodations or a native 
language version of the general 
assessment based on grade level 
standards. 

State demonstrates that LEP 
students are fully included in the 
State Accountability System. 


Accountability System. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



Currently, California EC Section 60640(b) requires all students in grades two through eleven to 
participate in the STAR assessment program. Similarly, California EC Section 60851(b) requires all grade 
ten students to participate in CAHSEE. English learners are not exempt from these requirements, but 
they may take the assessments with accommodations (see Supporting Evidence for Critical Element 5.3; 
additional regulations for accommodations for English learner students are under development). Test 
results from these students on the statewide assessment are summarized in the same fashion as any 
other student. 

In accord with state law, California will continue to test English learners during their first year of enrollment 
in United States schools; however, consistent with the flexibility offered by Secretary Paige’s 
communication of February 19, 2004, California elects to exclude the test results of these students from 
the AMO calculation. 

In determining the AYP of students with limited English proficiency, California will include the STAR and 
CAHSEE results of students who have been re-designated as fluent English proficient as well as English 
learners. In California, English learners are re-designated as fluent English proficient when they reach the 
“basic” level on the statewide ELA CST and the “proficient” level (referred to as early advanced) on 
California’s language proficiency test, called the California English Language Development Test 
(CELDT). The exit criteria for English learners include not only CELDT results but also local indicators 
defined by individual school districts as well as parental consultation. 

Because English learners are re-designated under state law and removed from the English learner 
subgroup before they attain the proficient level, re-designated fluent English proficient (RFEP) students 
must continue to be included in the English learner subgroup in order to demonstrate academic 
proficiency on the ELA CST. It is likewise impossible to provide a record of the success or failure that a 
school has with increasing the academic achievement level of these students. 

California will maintain RFEP students as English learners for purposes of NCLB until these students 
have attained the proficient level on the ELA CST for three years consistent with the federal definition of 
limited-English-proficient students in paragraph (25) of Section 91 01 of Title IX of NCLB. That definition 
includes students as limited English proficient when those students have “difficulties in speaking, reading, 
writing, or understanding the English language [that] may be sufficient to deny the individual - the ability 
to meet the State’s proficient level of achievement on State assessments described in section 
1111 (b)(3).” (Emphasis added.) California’s ELA standards require a student to develop a high level of 
academic achievement in English. California ensures that RFEP students will be successful on its high 
academic standards in English by continuing to provide high-quality instructional programs to RFEP 
students and requiring that RFEP students demonstrate sustained academic proficiency before exiting 
the English learner subgroup. Sustained academic proficiency requires three years of scoring proficient 
on the ELA CST to determine that a valid pattern of success has been established. Since RFEP students 
have already attained English language proficiency on the CELDT, there is no need to continue to 
administer the language proficiency assessment to the RFEP students. 

Supporting Evidence: 

• California Education Code Section 60640(b) that describes the inclusion of all students in the 
STAR program 

• California Education Code Section 60851 (b) that describes the requirements for all students 
to participate in the CAHSEE 

• California Code of Regulations Section 853(d) that describes California’s policy for testing 
students with limited English proficiency 

• Title IX, Section 91 01 (25)(D)(iii) of the No Child Left Behind Act of 2001 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.5 What is the State's 

definition of the minimum 
number of students in a 
subgroup required for 
reporting purposes? For 
accountability purposes? 


State defines the number of 
students required in a subgroup 
for reporting and accountability 
purposes, and applies this 
definition consistently across the 
State. 5 

Definition of subgroup will result in 
data that are statistically reliable. 


State does not define the required 
number of students in a subgroup 
for reporting and accountability 
purposes. 

Definition is not applied 
consistently across the State. 

Definition does not result in data 
that are statistically reliable. 



5 The minimum number is not required to be the same for reporting and accountability. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California defines 1 1 as the minimum number of students required to report subgroup results. This 
number has been selected as a result of confidentiality concerns (see Critical Element 5.6). California 
further will define the minimum number of students in a subgroup for accountability purposes to be: 

• 1 00 students with valid test scores, or 

• 50 students in those cases where the subgroup constitutes at least 1 5 percent of the students at 
the school with valid test scores. 

This would apply to schools, LEAs, and student subgroups within an LEA or school. The state criteria for 
determining numerically significant student subgroups were aligned with this definition effective January 
1,2005. 

Since the inception of its accountability system in 1999, California has required that numerically 
significant student subgroups in a school demonstrate “comparable improvement” in order for a school to 
be eligible for awards or meet its API growth target. As a result, California has had three years 
experience in establishing what constitutes an appropriate minimum subgroup size, including unexpected 
consequences that result from such a criterion. 

After analyzing the interaction between school-level and student subgroup accountability, California finds 
that there are three major considerations in establishing a minimum subgroup size. Any minimum “n” 
formula must strike a balance between these three: 

• The formula should reflect acceptable standards of statistical reliability. 

• The formula should encourage LEAs and schools to address the instructional needs of 
numerically significant ethnic and SED student subgroups. 

• The formula should take into account the use of the subgroup analysis, i.e., the consequential 
validity of the decision about whether or not the subgroup made AYP. 

A statistical analysis reveals that a minimum “n” of 100 for a subgroup that has 50 percent of its students 
at the “proficient” level or above still implies a standard error of 5 percent. Moreover, this error is 
compounded in the case of schools with diverse student populations because of the cumulative effect of 
repeated AYP decisions for each significant subgroup. From the standpoint of consequential validity, it is 
difficult to support a minimum “n” size under 1 00, especially for schools as diverse as those in California. 

In this connection, it is imperative to consider the consequential aspects of a given formula for minimum 
“n” size, or in other words, the likelihood that a given formula will yield reliable decisions about whether 
schools have actually made AYP, particularly in light of the high stakes attached to such a measure. 

On the other hand, a minimum “n” of 100, although technically meritorious, does not fully address the 
diversity of California’s schools in enrollment, student demographics, and geography. The adoption of 
this minimum “n” has the following implications for California’s schools: 

• It would result in a far larger number of schools for which an annual AYP decision is difficult if not 
impossible to make because the schools do not have a minimum of 100 valid test scores. 

• It may be logical for a small elementary school to design an instructional strategy to address the 
learning needs of a subgroup of less than 100, however illogical that may be for a large high 

school where such a group may represent a very small proportion of the overall student 
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population. 

• Using a one “minimum-‘n’-fits-all-schools” formula may have serious unintended consequences. 
For example, it could potentially result in the over-identification of large, urban high schools for 
program improvement, potentially resulting in an inappropriate redirection of school improvement 
resources away from elementary and middle schools. This would be counter-productive, since it 
is logical to assume that instructional changes that are implemented in elementary schools are 
fundamental in raising the achievement levels in high schools. 

Because these policy and instructional considerations are very important in California, the rule of 100 
students or 50 students and 1 5 percent of the students at the school with valid test scores is more 
appropriate and will result in a more accurate identification of California schools that fail to make AYP 
than a simple minimum “n.” Additional supporting evidence and data simulations will be presented to the 
peer review team. 

Supporting Evidence: 

• Council of Chief State Schools Officers (CCSSO) publication “Making Valid and Reliable 
Decisions in Determining AYP” (see pages 23 and 62-72) 

• California State Board of Education minutes from its January 8, 2003 meeting 

• Average Number of Significant Subgroups under California’s Minimum N Rule (Attachment O) 

• Average Number of Significant Subgroups by School Type (Attachment P) 

• “The Chances of False Negatives: Failing AYP Because of Random Processes: An Issue 
Paper" (Attachment Q) 

• “False Negatives with New AYP Definition: An Issue Paper” (Attachment R) 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


5.6 How does the State 
Accountability System 
protect the privacy of 
students when reporting 
results and when 
determining AYP? 


Definition does not reveal 
personally identifiable 
information. 6 


Definition reveals personally 
identifiable information. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


As per current practice, the CDE will not report test results for fewer than 1 1 students. AYP 
determinations for grades two through eight will be made and reported as a result of the summary of data 
across grade levels, further ensuring that student confidentiality will be maintained. 

Supporting Evidence: 

• California Code of Regulations Section 854 that describes California’s oolicv for reportina test 
scores 



6 The Family Education Rights and Privacy Act (FERPA) prohibits an LEA that receives Federal funds 
from releasing, without the prior written consent of a student’s parents, any personally identifiable 
information contained in a student’s education record. 
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PRINCIPLE 6. State definition of AYP is based primarily on the State’s academic 
assessments. 



CRITICAL ELEMENT 



6.1 How is the State’s 
definition of adequate 
yearly progress based 
primarily on academic 
assessments? 



EXAMPLES FOR 
MEETING REQUIREMENTS 



Formula for AYP shows that 
decisions are based primarily on 
assessments. 7 

Plan clearly identifies which 
assessments are included in 
accountability. 



EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 



Formula for AYP shows that 
decisions are based primarily on 
non-academic indicators or 
indicators other than the State 
assessments. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



AYP will be based primarily on the percentage of students at or above the “proficient” level in ELA and 
mathematics. This will be determined through the CSTs, the CAHSEE and the CAPA, as applicable. All 
three tests are based on the California content standards. The API, which functions as an additional 
indicator, is currently based exclusively on academic assessment results, both standards-based and 
norm-referenced. California EC Section 52052(a)(3) requires the eventual inclusion of attendance rates 
and graduation rates in the API when the rates are proven valid and reliable; however, it also requires test 
results to constitute at least 60 percent of the API. The only non-assessment component of AYP is the 
graduation rate, which is required by NCLB. 

Supporting Evidence: 

• California’s Public Schools Accountability Act of 1999 (PSAA) that describes the regulations 
guiding the composition of the Academic Performance Index (see Section 52052(a)(3)(A)) 

• 2002 Base API information that describes the current composition of the API 



7 State Assessment System will be reviewed by the Standards and Assessments Peer Review Team. 
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PRINCIPLE 7. State definition of AYP includes graduation rates for public High 
schools and an additional indicator selected by the State for public Middle and 
public Elementary schools (such as attendance rates). 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.1 What is the State definition 
for the public high school 
graduation rate? 


State definition of graduation rate: 

• Calculates the percentage 
of students, measured 
from the beginning of the 
school year, who graduate 
from public high school 
with a regular diploma (not 
including a GED or any 
other diploma not fully 
aligned with the state’s 
academic standards) in 
the standard number of 
years; or, 

• Uses another more 
accurate definition that 
has been approved by the 
Secretary; and 

• Must avoid counting a 
dropout as a transfer. 

Graduation rate is included (in the 
aggregate) for AYP, and 
disaggregated (as necessary) for 
use when applying the exception 
clause 8 to make AYP. 


State definition of public high 
school graduation rate does not 
meet these criteria. 



See USC 631 l(b)(2)(I)(i), and 34 C.F.R. 200.20(b) 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California does not currently have a student data system that would allow longitudinal tracking of 
individual students from grade nine or ten through high school graduation. Such a system is currently 
under development and once fully implemented can and will be used to calculate a high school 
graduation rate. Until that time, however, the CDE will use a synthetic graduation rate corresponding to 
the National Center for Educational Statistics (NCES) four-year completion rate. This rate includes 
information on high school completers and high school dropouts aggregated over a four-year period. 

This calculation methodology is advocated by the NCES in part so that there is consistent reporting 
across states. This methodology has also been advocated by the ED for use in the Consolidated State 
Application. Specifically, EDs Performance Goal 5 (all students will graduate from high school) states that 
“the percentage of students who graduate from high school each year with a regular diploma’’ should be 
calculated in the same manner as used in the NCES reports on the Common Core of Data. 

A change to the CBEDS data collection form has been implemented to make our state definition of 
“dropouts” consistent with that of NCES. The number of graduates, defined as the number of students 
who graduate from high school with a regular diploma, is collected via the CBEDS data collection form. 
This definition excludes students who receive a California high school equivalency certificate by passing 
the General Educational Development (GED) test. 

The four-year completion rate formula is shown below: 

HS CMP Yr. 4/ {HS CMP Yr. 4 + (Gr. 9 DPO Yr. 1 + Gr. 1 0 DPO Yr. 2 + Gr. 1 1 DPO Yr. 3 + Gr. 1 2 DPO 
Yr. 4)} 

HS = High School 
CMP = Completer 
Yr. = Year 
Gr. = Grade 
DPO = Dropouts 

Until California has established a longitudinal student tracking system, it is impossible to establish a four- 
year graduation rate for grade ten through twelve high schools. In the interim, California will establish a 
three-year rate with the formula adjusted accordingly. While it is improper to compare a four-year rate for 
grade nine through twelve schools to a three-year rate for grades ten through twelve schools, it is suitable 
for making longitudinal comparisons for the same school, so long as the school does not change grade 
spans. In the event that the school does change grade spans, then it is possible to adjust the rate by 
considering only the common grades from one year to the next. Therefore, this rate can be used to 
measure progress from one year to the next. The CDE has established a methodology to provide a four- 
year rate for grade ten through twelve schools. 

California began collecting graduate and dropout data for all the required student subgroups in 2004. Until 
four years of data are available, California will be unable to make use of the indicator in safe harbor 
calculations; however, it will still use the API for high schools to apply safe harbor. 

To demonstrate progress on the graduation rate, an LEA or school shall: 

a. Demonstrate an increase of one-tenth of a percentage point in the graduation rate, or 

b. Demonstrate an increase of two-tenths of a percentage point in the graduation rate when the 
average rate of the last available two years is compared with the average rate of the preceding 
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two years, or 

c. Meet an annual status target that begins at 82.8 percent (computed in a similar manner to the 
beginning AMOs in ELA and mathematics) and increases at a rate similar to the schedule of 
AMOs in ELA and mathematics. 

Proxy graduation rates 

For traditional comprehensive high schools without a graduating class, the CDE will calculate a proxy 
graduation rate using available CBEDS drop-out and enrollment data. This provision applies only to 
traditional comprehensive high schools; it does riot impact current procedures for assigning district or 
county rates to high schools with a primary mission of returning students to a traditional classroom 
environment, e.g., some continuation and alternative high schools. 

The proxy graduation rate will be calculated by first dividing the number of dropouts in all of the grades in 
the school (grades 9, 10, and 11) by the enrollment in the same grades. This percentage is then 
multiplied by four if the school enrolls 9 th graders only, by two if 9~ h and 10 th graders, or by 4/3 if 9 th , 10 th , 
and 1 1 th graders. The result approximates the percentage of students that would have dropped out if the 
school had enrolled students in all four grades (9,10,11, and 1 2). This percentage is then subtracted 
from 100 to approximate the graduation rate. 

An example of the calculation is as follows: 

In the first year of operation, a comprehensive high school enrolls ninth graders only. Each year it will add 
a grade. Therefore, it will not graduate students until its fourth year of operation. The ninth grade 
enrollment totals 300 students, five of whom drop out in the first year. 

The proxy graduation rate for this school would be: 

1 00% - ((5/300 x 1 00) x 4) = 1 00% - 6.6% = 93.4% 

Assignment of district or county rates 

The following rules will be applied only for high schools with a primary mission of returning students to a 
regular classroom environment in a comprehensive high school: 

• For high schools administered by a district, the CDE will assign them the value of the LEA 
graduation rate. 

• For direct-funded charter high schools, the CDE will assign the graduation rate of the charter 
authorizer. In cases where the charter authorizer does not have a graduation rate, the countywide 
graduation rate of the county in which the school is located will be assigned. 

• For high schools administered by county offices of education, the CDE will assign the 
countywide graduation rate. 

This approach is similar to the practice of assigning LEA values for the percent proficient or above to 
schools that serve grades outside those assessed. 

PI Identification 

California will identify for PI any high school that fails to meet the graduation rate criteria for AYP for two 
consecutive years. California will identify for PI any district with high school students if the district fails to 
meet the graduation rate criteria for two consecutive years. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.2 What is the State’s 
additional academic 
indicator for public 
elementary schools for the 
definition of AYP? For 
public middle schools for 
the definition of AYP? 


State defines the additional 
academic indicators, e.g., 
additional State or locally 
administered assessments not 
included in the State assessment 
system, grade-to-grade retention 
rates or attendance rates. 9 


State has not defined an 
additional academic indicator for 
elementary and middle schools. 




An additional academic indicator 
is included (in the aggregate) for 
AYP, and disaggregated (as 
necessary) for use when applying 
the exception clause to make 
AYP. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California proposes to use the API as the required additional academic indicator for elementary and 
middle schools as well as an optional academic indicator for high schools. Progress on that indicator will 
be defined as growth of at least one API point or meeting an annual API status target as defined by a 
schedule of API targets established by a methodology similar to the one used to establish the AMOs in 
ELA and mathematics. Disaggregated API results will be used when applying the exception clause (“safe 
harbor”) to make AYP. 

While the API includes the same test results used to determine AYP, it: 

• Reflects movement across four cut points, instead of only one 

• Includes norm-referenced test results in addition to standards-based test result 

• Includes additional content areas and additional standards tests at the high school level 

Therefore, the API is an ideal supplement to the AYP method of measuring academic performance, i.e., 
“proficient” or above in ELA and mathematics. 

Supporting Evidence: 

• API information (see Critical Element 1.1) 

• The Academic Performance Index (API) as the “Other” Indicator (see Attachment A) 

• The Academic Performance Index (API): A Six-Year Plan for Development (2001-2006) 



9 NCLB only lists these indicators as examples. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 

NOT MEETING REQUIREMENTS 


7.3 Are the State’s academic 
indicators valid and 
reliable? 


State has defined academic 
indicators that are valid and 
reliable. 

State has defined academic 
indicators that are consistent with 
nationally recognized standards, if 
any. 


State has an academic indicator 
that is not valid and reliable. 

State has an academic indicator 
that is not consistent with 
nationally recognized standards. 

State has an academic indicator 
that is not consistent within grade 
levels. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


California requires that the test publishers, with whom we have contracted, ensure that statewide 
assessments meet nationally recognized standards of validity and reliability. These assessments are the 
basis of both the AYP determination and the API calculation. 

Supporting Evidence: 

• Copy of a letter from the Educational Testing Service, California’s contractor for the CSTs, 
indicating it is ETS’s responsibility to ensure the reliability and validity of the statewide 
assessments (see Attachment S) 
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PRINCIPLE 8. AYP is based on reading/language arts and mathematics 
achievement objectives. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


8.1 Does the state measure 
achievement in 
reading/language arts and 
mathematics separately for 
determining AYP? 


State AYP determination for 
student subgroups, public 
schools and LEAs separately 
measures reading/language arts 
and mathematics. 10 

AYP is a separate calculation for 
reading/language arts and 
mathematics for each group, 
public school, and LEA. 


State AYP determination for 
student subgroups, public 
schools and LEAs averages or 
combines achievement across 
reading/language arts and 
mathematics. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


California currently reports assessment results separately by academic content area, including ELA and 
mathematics, on the CSTs, the CAHSEE, and the CAPA. AYP for a school and LEA will be determined 
by evaluating performance in ELA and mathematics separately. 


Supporting Evidence: 

• Sample school-level AYP report (see Attachment T) 





10 If the state has more than one assessment to cover its language arts standards, the State must create 
a method for including scores from all the relevant assessments. 
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PRINCIPLE 9. State Accountability System is statistically valid and reliable. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


9.1 HowdoAYP 

determinations meet the 
State’s standard for 
acceptable reliability? 


State has defined a method for 
determining an acceptable level of 
reliability (decision consistency) 
for AYP decisions. 

State provides evidence that 
decision consistency is (1) within 
the range deemed acceptable to 
the State, and (2) meets 
professional standards and 
practice. 

State publicly reports the estimate 
of decision consistency, and 
incorporates it appropriately into 
accountability decisions. 

State updates analysis and 
reporting of decision consistency 
at appropriate intervals. 


State does not have an 
acceptable method for 
determining reliability (decision 
consistency) of accountability 
decisions, e.g., it reports only 
reliability coefficients for its 
assessments. 

State has parameters for 
acceptable reliability; however, 
the actual reliability (decision 
consistency) falls outside those 
parameters. 

State’s evidence regarding 
accountability reliability (decision 
consistency) is not updated. 


STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 


As noted in Critical Element 5.5, statistical reliability is not necessarily the best measure of precision in 
making AYP determinations. Instead, the notion of consequential validity is of more importance, 
especially in light of the high stakes associated with the misclassification of schools. 

In this regard, the CDE currently supports research by Dr. David Rogosa of Stanford University on the 
potential for misclassification of schools based on API gains and losses. This includes an ongoing 
estimate of decision consistency. The CDE plans to extend this research to encompass the new federal 
AYP criteria as they are incorporated into the state accountability system, including the publication of 
estimates of decision consistency. 

Supporting Evidence: 

• “Accuracy of API Index and School Base Report Elements” bv Dr. David Roaosa (Dec. 2002) 

• “Year 2000 Update: Accuracy of API Index and School Base Report Elements” bv Dr. David 
Roaosa (December 2002) 

• “Year 2001 Update: Accuracy of API Index and School Base Report Elements” bv Dr. David 
Roaosa (December 2002) 

• “Irrelevance of Reliability Coefficients to Accountabilitv Systems: Statistical Disconnect in Kane- 
Staiaer ‘Volatility in School Test Scores’” bv Dr. David Roaosa (October 2002) 
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EXAMPLES FOR EXAMPLES OF 

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING 

REQUIREMENTS 

9.2 What is the State's process State has established a process State does not have a system for 

for making valid AYP for public schools and LEAs to handling appeals of accountability 

determinations? appeal an accountability decision, decisions. 



STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California plans to continue its already existing process for public schools and LEAs to appeal an AYP 
decision. Currently, annual appeals of the AYP decision can be made based on statistical or other 
substantive reasons ([per Title I, Part A, Section 1 1 16(b)(2)]). Appeals are required to be submitted to the 
CDE 10 days after an AYP determination is made. The CDE reviews each appeal on a case-by-case 
basis. 

In addition to an appeal of an AYP decision, following the publication of preliminary results in the summer, 
an LEA may also submit a request to correct erroneous demographic information that may affect the 
subgroup analysis on the statewide assessments used to determine AYP. This is in accord with 
established practice in conjunction with the API reporting cycle. 
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CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


9.3 How has the State planned 
for incorporating into its 
definition of AYP 
anticipated changes in 
assessments? 


State has a plan to maintain 
continuity in AYP decisions 
necessary for validity through 
planned assessment changes, 
and other changes necessary to 
comply fully with NCLB. 11 

State has a plan for including new 
public schools in the State 
Accountability System. 

State has a plan for periodically 
reviewing its State Accountability 
System, so that unforeseen 
changes can be quickly 
addressed. 


State’s transition plan interrupts 
annual determination of AYP. 

State does not have a plan for 
handling changes: e.g., to its 
assessment system, or the 
addition of new public schools. 



11 Several events may occur which necessitate such a plan. For example, (1) the state may need to 
include additional assessments in grades three through eight by 2005-2006; (2) the State may revise 
content and/or academic achievement standards; (3) the State may need to recalculate the starting point 
with the addition of new assessments; or (4) the State may need to incorporate the graduation rate or 
other indicators into its state accountability system. These events may require new calculations of validity 
and reliability. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California has a plan that describes the anticipated changes in the statewide assessments over the next 
three years. The three-year assessment plan closely interacts with the six-year plan for the API, because 
the API is based on the statewide assessments. (Recall the API is the additional indicator for AYP 
allowed under NCLB.) 

The API is computed in base-growth cycles. That is, for the 2002-03 cycle the base is computed on the 
2002 assessment results and released in January 2003; the growth is computed on 2003 assessment 
results and released in October 2003. The API calculation is always the same within each base-growth 
cycle (i.e. progress is assessed on the same indicators from Time 1 to Time 2). Any new assessments or 
new calculation methods are introduced into the API at the start of a new base-growth cycle. When the 
API was first developed in 1999 it included only a norm-referenced test. In 2001 the CST in ELA was 
added to the API. In 2002, the CST in mathematics and results from the CAHSEE were added. In 2003, a 
grade eight CST in history/social science and results from the CAPA were added. A grade five standards- 
based science assessment was added in 2004, and a grade eight and grade ten standards-based 
science assessment will be added in 2006 (science assessments are required under NCLB). 

One of the more immediate changes in our assessments that will impact AYP involves the establishment 
of three performance levels on the CAHSEE per the NCLB requirements. As described earlier in this 
workbook, a starting point for grades ten through twelve has now been calculated. This starting point was 
an estimate because when the CAHSEE was administered for the first time in 2001 it was offered on a 
voluntary basis to grade nine students. 

The SBE will annually review the plans for the statewide assessment system and the API. The CDE will 
expand this to include changes in the AYP measure resulting from changes in assessments as 
necessary. 

Supporting Evidence: 

• Three-year Plan for the Development of California’s Assessment System 

• The Academic Performance Index (API): A Six-Year Plan for Development (2001-2006) 
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PRINCIPLE 10. In order for a public school or LEA to make AYP, the State 
ensures that it assessed at least 95% of the students enrolled in each subgroup. 



CRITICAL ELEMENT 


EXAMPLES FOR 
MEETING REQUIREMENTS 


EXAMPLES OF 
NOT MEETING 
REQUIREMENTS 


1 0.1 What is the State's method 
for calculating participation 
rates in the State 
assessments for use in AYP 
determinations? 


State has a procedure to determine 
the number of absent or untested 
students (by subgroup and 
aggregate). 

State has a procedure to determine 
the denominator (total enrollment) 
for the 95% calculation (by 
subgroup and aggregate). 

Public schools and LEAs are held 
accountable for reaching the 95% 
assessed goal. 


The state does not have a 
procedure for determining 
the rate of students 
participating in statewide 
assessments. 

Public schools and LEAs are 
not held accountable for 
testing at least 95% of their 
students. 
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STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



The CDE is able to calculate participation rates because it requires that a SAD be completed for each 
student in the grades to be assessed, including students who do not take the assessment. 

Participation rates for AYP purposes will be calculated by dividing the number of students tested by the 
number of students enrolled on the first day of testing. Standard rounding rules will be applied in the 
calculation of participation rates. These rates will be calculated for ELA and mathematics separately. An 
LEA, school, or a numerically significant subgroup must have a participation rate of 95 percent or greater 
in order for an LEA, school, or a numerically significant subgroup to make AYP. 

In order to comply with the requirements of NCLB, for STAR the number of students enrolled on the first 
day of testing will include students who have been excluded from testing at the request of parents or 
guardians. This is contrary to past State practice in which these students were subtracted from the 
denominator in calculating participation rates. The right for a parent or guardian to request that a student 
be excused from STAR is recognized by California EC Section 60615. There are no similar provisions for 
student exclusions on the CAHSEE. Students participating in the CAPA will be included in both the 
numerator and denominator. 

California will consider all students who have sat for the assessment as participants. This includes 
students who failed to respond to enough items to generate a result. In testing parlance, these students 
are termed “did not attempts.” For accountability purposes, these latter students will be considered to be 
in the lowest performance level, i.e., “far below basic” and therefore as not proficient. 

Calculating participation rates for small schools/LEAs presents two major challenges: 

1 . Because of small numbers of students, these rates are subject to extreme fluctuation. 

2. A small school or LEA in effect has to exceed the 95 percent requirement since it is arithmetically 
impossible to achieve precisely a 95 percent participation rate. For example, a school with 50 
students would actually have to achieve a 96 percent participation rate. 

Assistant Secretary Simon addressed these issues in his letter to the Chief State School Officers on May 
1 9, 2004, and his letter to California on July 23, 2004. These letters linked the requirement for calculating 
the participation rate to the minimum size of the student subgroup. In California, to be considered 
numerically significant, a student subgroup must consist of at least 50 students who constitute 1 5 percent 
of the students in the grades assessed. A student subgroup of 100 or more students is considered 
significant even if it does not constitute 15 percent of the total number of students. 

Consistent with the flexibility outlined in the letters from Assistant Secretary Simon, California will: 

• Not calculate participation rates for schools/LEAs with less than 50 students eligible for 
testing, the lower boundary of California’s formula for calculating whether student subgroups 
are numerically significant or not. 

o Round the participation rate ujd to the nearest whole number for schools/LEAs or student 

subgroups with between 51 and 99 students eligible for testing. 

o Set a maximum of three non-tested students for schools/LEAs or student subgroups with 

precisely 50 students. (Otherwise, such a school or LEA would be held to a more rigorous 
criterion than a school or LEA with 51 students.) 
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Also, California elects to exercise the flexibility offered by Assistant Secretary Simon on May 1 9, 2004, 
and average school/LEA and subgroup participation rates for two or three years beginning in 2004. In its 
averaging methodology, California will aggregate participation data from two or three years and then 
apply the same procedures as it would for evaluating data from one year. 

For schools/LEAs and student subgroups with 100 or more students enrolled in the grades tested, 
California will employ standard rounding rules, as indicated above. 

Supporting Evidence: 

• Public Schools Accountability Act of 1999 Section 10310) that describes minimum participation 
rates for awards eligibility (see Title 5, Division 1 , Chapter 2, Subchapter 4, Article 1 .7, Section 
1 032(i)) 

• California Education Code Section 60615 that describes the allowance for parental waivers to 
excuse students from the statewide testing program 
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EXAMPLES FOR 


EXAMPLES OF 


CRITICAL ELEMENT 


MEETING REQUIREMENTS 


NOT MEETING 
REQUIREMENTS 


10.2 What is the State's policy 


State has a policy that 


State does not have a procedure 


for determining when the 


implements the regulation 


for making this determination. 


95% assessed 


regarding the use of 95% 




requirement should be 


allowance when the group is 




applied? 


statistically significant according 
to State rules. 





STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS 



California will apply the 95 percent requirement to any LEA, school, or student subgroup that meets its 
criteria for numerically significant in terms of enrollment in the grades tested . This will occur even if the 
LEA, school, or student subgroup does not generate enough valid scores to require determining whether 
or not the LEA, school, or student subgroup met the AMOs (see critical elements 3.2b and 3.2c). 

The 95 percent requirement is applied prior to the determination of whether or not the LEA, school, or 
student subgroup would have met the AMOs. An LEA, school, or student subgroup that does not meet 
the 95 percent requirement is deemed to have not made AYP, regardless of whether or not it would have 
met the AMOs, i.e. , the percent proficient or above. 



59 





CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK 



Appendix A 

Required Data Elements for State Report Card 



1111(h)(1)(C) 

1 . Information, in the aggregate, on student achievement at each proficiency level on the State academic 
assessments (disaggregated by race, ethnicity, gender, disability status, migrant status, English 
proficiency, and status as economically disadvantaged, except that such disaggregation shall not be 
required in a case in which the number of students in a category is insufficient to yield statistically 
reliable information or the results would reveal personally identifiable information about an individual 
student. 

2. Information that provides a comparison between the actual achievement levels of each student 
subgroup and the State’s annual measurable objectives for each such group of students on each of 
the academic assessments. 

3. The percentage of students not tested (disaggregated by the student subgroups), except that such 
disaggregation shall not be required in a case in which the number of students in a category is 
insufficient to yield statistically reliable information or the results would reveal personally identifiable 
information about an individual student. 

4. The most recent 2-year trend in student achievement in each subject area, and for each grade level, 
for the required assessments. 

5. Aggregate information on any other indicators used by the State to determine the adequate yearly 
progress of students in achieving State academic achievement standards disaggregated by student 
subgroups. 

6. Graduation rates for secondary school students disaggregated by student subgroups. 

7. Information on the performance of local educational agencies in the State regarding making adequate 
yearly progress, including the number and names of each school identified for school improvement 
under section 1116. 

8. The professional qualifications of teachers in the State, the percentage of such teachers teaching with 
emergency or provisional credentials, and the percentage of classes in the State not taught by highly 
qualified teachers, in the aggregate and disaggregated by high-poverty compared to low-poverty 
schools which (for this purpose) means schools in the top quartile of poverty and the bottom quartile of 
poverty in the State. 
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